Visual effects generation can involve one or more of a plurality of different system components, inclusive of cameras, lighting, recording systems, synchronization devices, wearable devices, markers, virtual devices, and viewing devices, among others. Visual effects have also played an increasing role in modern films, including use of various visual effects components in virtual production.
Virtual Production is the means, system, or set of processes by which computer-generated three dimensional environments, objects, and characters are integrated with a motion-capture based data record of movement, dimensions and spatial relationships.
Virtualization is accomplished by generalist means that vary according to the needs of the production, and may be suited to fully virtualized or hybridized productions that integrate live action photography, utilizing virtual cinematography, virtual camera operation, etc.
PRIOR ART FIG. 1 illustrates a comparison flow diagram of a traditional filmmaking process, shown generally at 12, and an exemplary virtual production process, shown generally at 14. Development in the context of traditional filmmaking 12 begins with a story, shown at 16, progresses to a preparation stage including activity by an art department, selection of locations and construction of sets, shown at 18, to production of a live action shoot, shown at 20, and to post production editing, shown at 22, and resultant movie production at 23. The illustrated FIGURE also illustrates possible roles for third party vendors, shown generally at 24, pulling and working with production material from the live action shoot at 20, as well as from post production editing at 22 to contribute to the post production editing.
While individual aspects will be described in more detail below, the exemplary illustrated comparative virtual production flow diagram 14 also begins with a development phase including generation of a story by one or both of a director and writer, shown at 26. A preparation phase includes a production designer and art (ART) department, shown at 28, with possible earlier involvement of third party vendors, shown at 30, to prepare for and facilitate production, for example via early concept work that can be fed into production or post production editing.
In the production phase of the illustrated exemplary virtual production flow diagram, an exemplary virtual asset/art department (VAD), shown at 32, provides various inputs for virtual production itself. While aspects of VAD will be described in further detail below, VAD aspects and involvement in virtual production can vary, with exemplary roles including: acting as a concept source, as informed by production, to create 3D assets; turning 2D art concepts into virtual production ready 3D assets; making existing 3D assets production worthy (e.g., from a third party vendor or otherwise); and other roles including working with other teams in a virtual production pipeline to make 3D assets ready for use in the virtual production pipeline, including preparing assets for use in principal motion capture, virtual camera sessions, previsualization, post visualization, and animation, among others. Accordingly, VAD 32, feeds into production stage work, shown at 34, and production editing, shown at 36, and draws from such editing in a cyclical way, illustrated at virtual production flow arrow 38. An exemplary production phase may also include studio review, shown at 40. Live action shooting is illustrated at 41. Post production picture editing and a final movie generated from virtual production are also illustrated at 42 and 44, respectively.
Various films that represent advancements in virtual production include, for example: Avatar; A Christmas Carol; Tintin; Mars Needs Moms; Jack the Giant Slayer; various Marvel movies; Maleficent; various Planet of the Apes movies; The Jungle Book; Ready Player One; and the forthcoming Call of the Wild, among others.
Avatar provides one good example of the usefulness of various visual effects components in a virtual production. For example, PRIOR ART FIG. 2, which is an image that illustrates several components, including a motion capture suit 46, wearable markers 48 and a headrig 50, including a facial capture camera unit 52 mounted on a boom 54. The camera unit recorded the movement of the actor's face, providing relevant facial information such that CG artists could digitally render virtual characters.
PRIOR ART FIG. 3 is an image that shows plural actors in wearable motion capture suits and headrigs, with electrical tethers 56 extending away from the actors. Certain of the actors also carry props having markers thereon, in this case in the shape of bows. A worn camera recorder 60 can also be seen in the image, as well as a worn camera synchronization unit 62.
PRIOR ART FIG. 4 is an image that shows plural characters in a virtual production stage, or “volume”, with plural reference cameras 64 pointed at different actors shooting various shots, including close-ups, medium shots and wide angles of different characters in the scene to provide material for editors to take the best takes during performance editing.
PRIOR ART FIGS. 5 and 6 are images of a virtual camera that includes a stationary handle 66, a pivoting handle 68, markers 70 and a viewscreen 72. Tracked by the motion-capture system, it showed the director a real-time image of the virtual world, including actor's performances that were captured, even for pre-recorded scenes. Button controls (not shown) also allowed replication of live action set controls for the virtual camera, for example changing scales or executing aerial, dolly and crane moves.
Additionally, all recorded material could be managed in a pipeline or digital asset management system that could contain all of the artwork, track all of the performance capture takes and be the repository for all of the visual-effects work. Further reference is made, for a more complete description of the technology and techniques used for the virtual production of AVATAR, generally to “The Making of AVATAR”, Jody Duncan, Lisa Fitzpatrick, © 2010 Twentieth Century Fox Corporation, ISBN 978-0-8109-9706-6, from which PRIOR ART FIGS. 2-6 were drawn.
The present disclosure describes advancements in one or more of the above-described components and systems for virtual production.